feat(ci): add contributor reputation check workflow by imran-siddique · Pull Request #1520 · github/awesome-copilot

imran-siddique · 2026-04-27T05:09:12Z

Summary

Add automated contributor reputation screening on PR/issue open events to detect coordinated inauthentic contribution patterns (e.g., credential-laundering campaigns, spray-and-pray governance issues).

How it works

Checks out AGT's contributor reputation scripts via sparse-checkout
Runs profile check (account age, repo patterns, spray detection)
Runs credential audit (checks for merged-PR-to-citation pipelines)
Posts a comment and adds a label on MEDIUM/HIGH risk contributions
Skips bot accounts (dependabot, github-actions, copilot-swe-agent)

Why this matters

Multiple AI agent framework repos have been targeted by coordinated campaigns that:

Spray low-effort governance issues across 30+ repos in days
Get trivial PRs merged, then cite them as credentials in other repos
Use foundation submissions to build credibility for product placement

This workflow helps maintainers catch these patterns early.

Dependencies

Uses scripts from microsoft/agent-governance-toolkit (stdlib-only Python, no pip install needed)
Runs on pull_request_target and issues events only
No secrets beyond the default GITHUB_TOKEN

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

github-actions

⚠️ This PR targets main, but PRs should target staged.

The main branch is auto-published from staged and should not receive direct PRs.
Please close this PR and re-open it against the staged branch.

You can change the base branch using the Edit button at the top of this PR,
or run: gh pr edit 1520 --base staged

Copilot

Pull request overview

Adds a new GitHub Actions workflow to automatically screen new PR/issue authors using the Agent Governance Toolkit (AGT) and to surface elevated-risk contributors via comments and labels.

Changes:

Introduces .github/workflows/contributor-check.yml triggered on pull_request_target and issues opened events.
Runs two AGT Python checks (profile + credential audit) and computes an overall risk level.
Posts a PR/issue comment and applies a needs-review:<RISK> label for MEDIUM/HIGH outcomes.

aaronpowell · 2026-04-28T00:25:51Z

Please don't contribute new workflows and policies without discussing with the maintainers first.

aaronpowell · 2026-05-01T01:04:30Z

+  pull_request_target:
+    types: [opened]
+  issues:
+    types: [opened]


Have you experimented with running this on discussions too? Would that be useful to do?

aaronpowell · 2026-05-01T01:06:40Z

+      - name: Checkout AGT scripts
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          repository: microsoft/agent-governance-toolkit
+          sparse-checkout: scripts
+          path: agt


Do we have to clone the repo? Is it not possible to install the CLI with pip?

If we have to clone the repo, it might be a good idea to pin a tag or release, so that if there is drift in the script it doesn't result in this action failing.

aaronpowell · 2026-05-01T01:08:07Z

+          profile="${{ steps.profile.outputs.risk }}"
+          cred="${{ steps.credential.outputs.risk }}"
+
+          if [ "$risk" = "HIGH" ]; then icon="🔴"; else icon="🟡"; fi


Should low risk not have a separate icon?

aaronpowell · 2026-05-01T01:09:30Z

+          body="<!-- agt-contributor-check -->
+          $icon **Contributor Reputation Check: $risk risk**
+
+          | Check | Risk |


Ah, my comment on the discussion is probably irrelevant - I wrote that before reviewing the PR and I was assuming the PR was going to be using the action and Python script the action runs from the repo, not a custom script here.

aaronpowell · 2026-05-01T01:10:57Z

+            --repo "${{ github.repository }}" \
+            --json > /tmp/cred.json 2>/tmp/cred.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


It only extracts the risk property from the JSON, is there anything else from the JSON that might be useful to extract and display?

aaronpowell · 2026-05-01T01:15:32Z

+            --repo "${{ github.repository }}" \
+            --json > /tmp/cred.json 2>/tmp/cred.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


Suggested change

risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")

risk=$(cat '/tmp/cred.json' | jq '.risk // "UNKNOWN"')

Can simplify with jq rather than loading a Python environment for that.

aaronpowell · 2026-05-01T01:16:14Z

+            --username "${{ steps.author.outputs.username }}" \
+            --json > /tmp/profile.json 2>/tmp/profile.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


Suggested change

risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")

risk=$(cat '/tmp/profile.json' | jq '.risk // "UNKNOWN"')

Can simplify with jq rather than loading a Python environment for that.

aaronpowell · 2026-05-01T01:16:35Z

+            --username "${{ steps.author.outputs.username }}" \
+            --json > /tmp/profile.json 2>/tmp/profile.log
+          set -e
+          risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null || echo "UNKNOWN")


It only extracts the risk property from the JSON, is there anything else from the JSON that might be useful to extract and display?

…ommands (#1711) Add agt-contributor-check and agt-credential-audit as console_scripts entry points in the agent-governance-toolkit package. This allows consumers to pip install instead of cloning the repo to access these tools. Addresses: github/awesome-copilot#1520 review feedback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

imran-siddique · 2026-05-04T01:49:17Z

Addressed all review feedback in d49d70f:

pip install instead of cloning - Replaced repo checkout with \pip install agent-governance-toolkit==3.3.0. CLI entry points (\�gt-contributor-check, \�gt-credential-audit) just landed on AGT main (microsoft/agent-governance-toolkit#1711). This pins to a release version and eliminates the supply-chain concern.

jq instead of Python - Switched to \jq -r '.risk // "UNKNOWN"'. Simpler, no Python env needed for JSON parsing.

UNKNOWN maps to MEDIUM - Fixed. The workflow now fails closed: unknown/error states are treated as MEDIUM risk, not LOW.

Markdown indentation - Switched to heredoc so the comment renders correctly.

Unused variable - Removed.

Low risk icon - Added ✅ for LOW in the job summary.

More JSON data - Intentionally keeping only the risk level in the PR comment. The verbose output is by design: we surface the signal (risk level) and link to the workflow run for anyone who wants the full breakdown. This avoids leaking profile heuristic details that could help adversaries tune their patterns.

Discussions trigger - Good idea, will explore as a follow-up once we validate this on PRs/issues first.

@aaronpowell ready for re-review when you get a chance.

aaronpowell · 2026-05-04T02:05:51Z

I've changed the base branch of the PR, but because you branched from main it's still pretty messed up.

You can attempt to fix this with a rebase:

git fetch origin staged
git rebase --onto origin/staged origin/main <branch name>
git push --force-with-lease

If that does not resolve it, you can run npm run plugin:clean which will delete the materialised plugins and you can commit that change.

Add automated contributor reputation screening on PR/issue open events using AGT's pip-installable CLI tools. Detects coordinated inauthentic contribution patterns (credential laundering, spray-and-pray). - Installs via pip (pinned to agent-governance-toolkit==3.3.0) - Uses jq for JSON parsing - Fails closed: UNKNOWN risk maps to MEDIUM - Posts risk summary comment on MEDIUM/HIGH with link to workflow run - Adds needs-review label for maintainer attention Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

imran-siddique · 2026-05-04T03:59:53Z

Rebased onto \staged. PR is now a single clean commit with just the workflow file (1 file, +152 lines). Thanks for fixing the base branch.

…ommands (microsoft#1711) Add agt-contributor-check and agt-credential-audit as console_scripts entry points in the agent-governance-toolkit package. This allows consumers to pip install instead of cloning the repo to access these tools. Addresses: github/awesome-copilot#1520 review feedback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

imran-siddique requested a review from aaronpowell as a code owner April 27, 2026 05:09

Copilot AI review requested due to automatic review settings April 27, 2026 05:09

github-actions Bot requested changes Apr 27, 2026

View reviewed changes

Copilot started reviewing on behalf of imran-siddique April 27, 2026 05:09 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

Comment thread .github/workflows/contributor-check.yml Outdated

Comment thread .github/workflows/contributor-check.yml Outdated

Comment thread .github/workflows/contributor-check.yml Outdated

Comment thread .github/workflows/contributor-check.yml Outdated

aaronpowell closed this Apr 28, 2026

aaronpowell reopened this May 1, 2026

aaronpowell requested a review from dvelton as a code owner May 1, 2026 00:58

aaronpowell reviewed May 1, 2026

View reviewed changes

github-actions Bot force-pushed the main branch 6 times, most recently from f6ee187 to 44bd367 Compare May 4, 2026 01:24

imran-siddique mentioned this pull request May 4, 2026

feat(cli): expose contributor-check and credential-audit as pip CLI commands microsoft/agent-governance-toolkit#1711

Merged

github-actions Bot force-pushed the main branch from 44bd367 to f109701 Compare May 4, 2026 02:01

aaronpowell changed the base branch from main to staged May 4, 2026 02:03

imran-siddique force-pushed the feat/contributor-check branch from d49d70f to e5f900e Compare May 4, 2026 03:59

aaronpowell approved these changes May 4, 2026

View reviewed changes

aaronpowell merged commit c02894b into github:staged May 4, 2026
8 checks passed

	risk=$(python -c "import json; print(json.load(open('/tmp/cred.json'))['risk'])" 2>/dev/null \|\| echo "UNKNOWN")
	risk=$(cat '/tmp/cred.json' \| jq '.risk // "UNKNOWN"')

	risk=$(python -c "import json; print(json.load(open('/tmp/profile.json'))['risk'])" 2>/dev/null \|\| echo "UNKNOWN")
	risk=$(cat '/tmp/profile.json' \| jq '.risk // "UNKNOWN"')

Conversation

imran-siddique commented Apr 27, 2026

Summary

How it works

Why this matters

Dependencies

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aaronpowell commented Apr 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

imran-siddique commented May 4, 2026

Uh oh!

aaronpowell commented May 4, 2026

Uh oh!

imran-siddique commented May 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants